Differential contributions of body form, motion, and temporal information to subjective action understanding in naturalistic stimuli.

Introduction: We investigated the factors underlying naturalistic action recognition and understanding, as well as the errors occurring during recognition failures. Methods: Participants saw full-light stimuli of ten different whole-body actions presented in three different conditions: as normal videos, as videos with the temporal order of the frames scrambled, and as single static representative frames. After each stimulus presentation participants completed one of two tasks-a forced choice task where they were given the ten potential action labels as options, or a free description task, where they could describe the action performed in each stimulus in their own words. Results: While generally, a combination of form, motion, and temporal information led to the highest action understanding, for some actions form information was sufficient and adding motion and temporal information did not increase recognition accuracy. We also analyzed errors in action recognition and found primarily two different types. Discussion: One type of error was on the semantic level, while the other consisted of reverting to the kinematic level of body part processing without any attribution of semantics. We elaborate on these results in the context of naturalistic action perception.

A computational neuroethology perspective on body and expression perception.

Survival prompts organisms to prepare adaptive behavior in response to environmental and social threat. However, what are the specific features of the appearance of a conspecific that trigger such adaptive behaviors? For social species, the prime candidates for triggering defense systems are the visual features of the face and the body. We propose a novel approach for studying the ability of the brain to gather survival-relevant information from seeing conspecific body features. Specifically, we propose that behaviorally relevant information from bodies and body expressions is coded at the levels of midlevel features in the brain. These levels are relatively independent from higher-order cognitive and conscious perception of bodies and emotions. Instead, our approach is embedded in an ethological framework and mobilizes computational models for feature discovery.

Being the victim of virtual abuse changes default mode network responses to emotional expressions.

Recent behavioural studies have provided evidence that virtual reality (VR) experiences have an impact on socio-affective processes, and a number of findings now underscore the potential of VR for therapeutic interventions. An interesting recent result is that when male offenders experience a violent situation as a female victim of domestic violence in VR, their sensitivity for recognition of fearful facial expressions improves. A timely question now concerns the underlying brain mechanisms of these behavioural effects as these are still largely unknown. The current study used fMRI to measure the impact of a VR intervention in which participants experienced a violent aggression from the specific vantage point of the victim. We compared brain processes related to facial and bodily emotion perception before and after the VR experience. Our results show that the virtual abuse experience led to an enhancement of Default Mode Network (DMN) activity, specifically associated with changes in the processing of ambiguous emotional stimuli. In contrast, DMN activity was decreased when observing fully fearful expressions. Finally, we observed increased variability in brain activity for male versus female facial expressions. Taken together, these results suggest that the first-person perspective of a virtual violent situation impacts emotion recognition through modifications in DMN activity. Our study contributes to a better understanding of the brain mechanisms associated with the behavioural effects of VR interventions in the context of a violent confrontation with the male participant embodied as a female victim. Furthermore, this research also consolidates the use of VR embodied perspective-taking interventions for addressing socio-affective impairments.

Computation-Based Feature Representation of Body Expressions in the Human Brain.

Humans and other primate species are experts at recognizing body expressions. To understand the underlying perceptual mechanisms, we computed postural and kinematic features from affective whole-body movement videos and related them to brain processes. Using representational similarity and multivoxel pattern analyses, we showed systematic relations between computation-based body features and brain activity. Our results revealed that postural rather than kinematic features reflect the affective category of the body movements. The feature limb contraction showed a central contribution in fearful body expression perception, differentially represented in action observation, motor preparation, and affect coding regions, including the amygdala. The posterior superior temporal sulcus differentiated fearful from other affective categories using limb contraction rather than kinematics. The extrastriate body area and fusiform body area also showed greater tuning to postural features. The discovery of midlevel body feature encoding in the brain moves affective neuroscience beyond research on high-level emotion representations and provides insights in the perceptual features that possibly drive automatic emotion perception.

The role of computational and subjective features in emotional body expressions.

Humans are experts at recognizing intent and emotion from other people's body movements; however, the underlying mechanisms are poorly understood. Here, we computed quantitative features of body posture and kinematics and acquired behavioural ratings of these feature descriptors to investigate their role in affective whole-body movement perception. Representational similarity analyses and classification regression trees were used to investigate the relation of emotional categories to both the computed features and behavioural ratings. Overall, postural rather than kinematic features discriminated better between emotional movements for the computed as well as for the behavioural features. In particular, limb angles and symmetry appeared to be the most relevant ones. This was observed independently of whether or not the time-related information was preserved in the computed features. Interestingly, the behavioural ratings showed a clearer distinction between affective movements than the computed counterparts. Finally, the perceived directionality of the movement (i.e. towards or away from the observer) was found to be critical for the recognition of fear and anger.

Looking at the face and seeing the whole body. Neural basis of combined face and body expressions.

In the natural world, faces are not isolated objects but are rather encountered in the context of the whole body. Previous work has studied the perception of combined faces and bodies using behavioural and electrophysiological measurements, but the neural correlates of emotional face-body perception still remain unexplored. Here, we combined happy and fearful faces and bodies to investigate the influence of body expressions on the neural processing of the face, the effect of emotional ambiguity between the two and the role of the amygdala in this process. Our functional magnetic resonance imaging analyses showed that the activity in motor, prefrontal and visual areas increases when facial expressions are presented together with bodies rather than in isolation, consistent with the notion that seeing body expressions triggers both emotional and action-related processes. In contrast, psychophysiological interaction analyses revealed that amygdala modulatory activity increases after the presentation of isolated faces when compared to combined faces and bodies. Furthermore, a facial expression combined with a congruent body enhanced both cortical activity and amygdala functional connectivity when compared to an incongruent face-body compound. Finally, the results showed that emotional body postures influence the processing of facial expressions, especially when the emotion conveyed by the body implies danger.